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Consider an information source generating a symbol at the root of a tree network whose 
links correspond to noisy communication channels, and broadcasting it through the network. 
We study the problem of reconstructing the transmitted symbol from the information re- 
ceived at the leaves. In the large system limit, reconstruction is possible when the channel 
noise is smaller than a threshold. 

We show that this threshold coincides with the dynamical (replica symmetry breaking) 
glass transition for an associated statistical physics problem. Motivated by this correspon- 
dence, wc derive a variational principle which implies new rigorous bounds on the reconstruc- 
tion threshold. Finally, we apply a standard numerical procedure used in statistical physics, 
to predict the reconstruction thresholds in various channels. In particular, we prove a bound 
on the reconstruction problem for the antifcrromagnctic "Potts" channels, which implies, in 
the noiseless limit, new results on random proper colorings of infinite regular trees. 

This relation to the reconstruction problem also offers interesting perspective for putting 
on a clean mathematical basis the theory of glasses on random graphs. 

PACS numbers: 02.50.-r (Probability theory, stochastic processes, and statistics), 64.70.Pf (Glass 
transitions), 89. 75. He (Networks and genealogical trees) 

I. INTRODUCTION 



Consider the following broadcast problem 



MosOJ|. An information source at the root of a 



tree network produces a letter taken from a <?-ary alphabet x S {1, . . . , q} (we shall sometimes 
refer to a letter from this alphabet as to a 'color'). The symbol is propagated along the edges of 
the tree. For simplicity we start with a regular £>ary tree T^, cf. Fig. ^ in which every vertex 
has exactly k descendants (every verte x has deg ree k + 1 except the root which has degree k), 
a more general setting is described in jEKPSOfl ] and in Sect. I VI II Each edge of the tree is an 
instance of the same noisy communication channel: If the letter x is transmitted through the 



FIG. 1: Left: Broadcast on a tree. The signal is sent from the root. Each edge is a noisy communication 
channel broadcasting upwards. Right: The reconstruction problem asks to find what signal was sent from 
the root, given the signals received on the leaves 



channel, y £ {1, ...,(?} is received with probability ir(y\x) (with ir(y\x) > and ^2 y Tr(y\x) = 1). 
The problem of reconstruction is the following: consider all the symbols received at the vertices 
of the £ th generation. Does this configuration contain a non-vanishing information on the letter 
transmitted by the root, in the large £ limit? 

Beyond its fundamental i nterest i n probability, this problem is relevant to genetics (propagation 



of genes from an ancestor) 



MS05bJ, to statistical physics (models on Bethe lattices), and infor- 



mation t heory ( the problem being equivalent to computing the information capacity of the tree 
network) 



KS66b 



KS66al. Con- 



An important general bound was obtained by Kesten and Stigum (KS) 
sider the matrix tt with entries 7r(y\x), x,y 6 {1, ...,(?} and let A2(vr) be its eigenvalue with the 
second largest absolute value. Then, if A; | A2 | 2 > 1, the reconstruction problem is solvable: the 
leaves asymptotically contain some information on the letter sent by the root. In fact in this case 
the census of the variables in the £ th generation (the number of leaves which have received each 
letter) contains some information on the ro ot. Con versely, if A; | A2 (tt) | 2 < 1, the census contains 
asymptotically no information on the root Therefore, the KS condition A; | A2 (^) | 2 = 1 

defines a threshold for the maximum amount of noise allowing census reconstruction. For larger 
noise ( /c|A2(vr)| 2 < 1) one may wonder whether reconstruction is possible exploiting the whole 
set of symbols received at the £ th generation, through a clever use of the correlations between the 
symbols received on the leaves. The answer depends on the channel. 

In most of this paper we shall focus onto transition kernels vr( • | • ) satisfying the detailed balance 
condition (reversible) with respect to the uniform distribution r](x) = 1/q. In other words 7r(y\x) = 
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ir(x\y). With a slight abuse of notation we shall write 7r(y\x) = 7r(y, x). For the problem to be non- 
trivial, we also assume 7r( • | • ) to be irreducible and aperiodic. A particularly important example in 
this family is provided by (/-ary symmetric channels (or, borrowing from the statistical mechanics 
terminology, 'Potts' channels) 



vr(y|x) 



1 — e if y = x. 

(1) 

e/(q — 1) otherwise 



If e < 1 — 1/q, y = x is the most likely channel output when the input is x: we shall refer to 
this case as the 'ferromagnetic' Potts channel. If e > 1 — 1/q, the opposite happens and we shall 
speak of 'antiferromagnetic' Potts channel. The particular case e = 1 is of special interest, since 
the broadcast process provides a uniformly random proper coloring of the ^-generations /c-ary tree 
T k (l). 

It is intuitively clear that the channel (^Q) 'gets worse' as e increases from to 1 — 1/q (ferro- 
magnetic c hannel) and 'improves' as e goes from 1 — 1/q to 1 (antiferromagnetic channel). A result 



by Mossel 



MosOll ] implies that there exist a ferromagnetic and an antiferromagnetic threshold, 



respectively ef{k, q) £ [0, 1 — 1/q] and e~(k, q) G [1 — 1/q, 1], such that the reconstruction problem 
is solvable when e 6 [0, £+[ U ]e~, 1] and insolvable if e £ ]e^,e,r[- Hereafter we shall drop the ± 
superscripts whenever they are clear from the context. 

The KS condition A;| A2 (tt) | 2 > 1 is satisfied (and the problem is census-solvable) for the channel 
HJ if and only if e £ [0, £^{k, q)[ U }e^{k, q),l\, where: 



' 9) = ' L ~q' { lT 7=k ] 



Notice that the above formula yields e^ s (k,q) > 1 for some pairs of (k,q). In fact, for the 
antiferromagnetic channel, the census-reconstruction problem (as well as the general reconstruction 
problem) is not necessar ily solvable for e = 1. 



It is known 



BRZQ3] that, for q = 2 (the "binary symmetric" channel, also known as the 



"symmetric Ising" case), the reconstruction threshold is equal to the KS one: e T (k,2) = EKs(k,2) 
(for q = 2 the ferromagnetic and antiferromagnetic cases are equivalent via the mapping eh 1 — e) . 



In genera 
in 



the KS bound implies e^ s (k,q) > e^{k,q), and e~{k,q) > e^ s (k,q). Furthermore, 
MosOl] it was shown that, for all k, when q is large enough, e^ s (k,q) > e^(k,q) strictly: 
reconstruction is possible at noise levels where census reconstruction does not work. However, 
several fundamental questions remain open even for simple Potts channels: Is there any pair (k, q), 
with q > 2, such that e r (k,q) = eKs(k,q)^ How to distinguish systematically between e r (k,q) and 
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EKs(k, q)? How to determine e T (k, q) accurately when it does not coincide with £ks(^ 5 ?)? We shall 
address these issues in the following. 

The reconstruction problem is intimately related to statistical physics. Consider a model of 
Potts spins yi G \l,...,q}, on a finite rooted tree with t generations, to be denoted by 
Suppose that the energy of a configuration y l = {yi : i € Tk(i)} is given by: 

E{y l ) = -J E W (3) 
(i,j)er k (e) 

where denotes pairs of spins connected by an edge of the tree. Let Yf be the random 

configuration produced by the broadcast process with channel (^Q) up to generation I, when the 
transmitted symbol is uniformly random in {1, ... ,q}. Then 

P{Y! = y l } = ^exp{-(3E(]/)} (4) 
provided we make the identification 

In other words, the broadcast process allows to construct one particular Gibbs measure (state) 
associated to the energy function (j3J): in the statistical physics terminology this is the free-boundary 
measure. In general this is not the unique Gibbs measure for this energy function. For instance, 
if e < (l — j;), one can construct q 'ferromagnetic' states as well. Even if more than one 

Gibbs state exists, the free-boundary state can be extremal (or 'pure'). It turns out that the 
reconstruction problem is solvable if and only if the Gibbs state with free boundary conditions is 
not extremal. 

Gi ven the strong connection between extremality of Gibbs states and spatial decay of correla- 
tions [Geo88l ] . the last remark is not surprising. What is more surprising (and constitutes the main 
theme of this paper) is the relation of the reconstructibility with the existence of a dynamical glass 
phase. In recent years, an ongoing effort has been devoted to the study of glassy models on sparse 
random graphs. These are graphs which contain cycles but locally 'look like' a tree (e.g. uniformly 
random graphs with given degree). One of the most widespread features of these models, is the 
occurrence of glass phases in which the Boltzmann measure gets split into an exponential number of 
'lumps' (also referred to as clusters or pure states). This phenomenon is usually studied by solving 
some 'one-step replica symmetry breaking' (1RSB) distributional equations. In the following we 
show that these equations, as well as the criterion used to detect glass phases, do indeed coincide 
with the solvability of an appropriate reconstruction problem. 



5 



In spin glass theory, one can encounter two types of transitions to a glass phase. In the first case 
the transition is continuous in a properly defined order parameter. In spin glass jargon this leads 
to a phase with 'full replica symmetry breaking' (FRSB). In the second it is discontinuous, leading 
to 1RSB. Both situations occur in the reconstruction problem, depending on the alphabet and the 
channel. In the continuous case, the phase transition location is given by a local instability which 
coincides with the KS threshold, and one has e r (k, q) = e^sik, q). This happens, for instance, when 
q = 2. In the opposite case, the 'dynamical' glass transition is discontinuous and its location (which 
still coincides with the reconstruction threshold) is distinct from the KS one. In the ferromagnetic 
Potts model one has, for instance, e r (k,q) > EKs(k,q) at large enough q. 

The coincidence of the reconstruction threshold with the dynamical glass transition, apart from 
being interesting in itself, allows us to adapt several techniques developed within the theory of spin 
glasses in order to study the reconstruction problem. On the one hand, importing a numerical 
procedure currently used in this field, we determine the threshold for several pairs k, q. These 
results lead us to conjecture that e T (k,q) = EKs(k,q), for k not too large and q < 4 (in the 
ferromagnetic case) or q < 3 (in the antiferromagnetic case). 

Furthermore, we derive a variational principle for the reconstruction problem. In the antifer- 
romagnetic case, this implies a rigorous bound on the reconstruction threshold, which allows to 
confirm the strict inequality e~(k,q) < e^ s (k,q) in most of the cases in which this was found to 
be the case numerically. Although we conjecture such a bound to hold in much greater generality, 
we weren't able to prove it, and we leave it as a conjecture. 

The paper is organized as follows. In Section[n]we define the main objects studied in the paper 
and prove the coincidence between reconstruction and dynamical glass transition. In Section II I II 
we state our variational principle and prove that it provides a rigorous bound for a class of kernels 
7r( - 1 • ) including the antiferromagnetic model. In Sees. IIVI and Ivl we apply this principle as well 
as a numerical procedure to the determination of thresholds for the Potts channel, respectively in 
the ferromagnetic and antiferromagnetic case. Section IVII discusses the physical meaning of the 
relation between reconstruction and glass transitions. Section [VIII explains how our methods (and 
the glass - reconstruction correpondance) can be generalized to a broad category of broadcast and 
reconstruction problems on trees, going much beyond the Potts channel. We conclude in Section 
IVII II bv summarizing a few conjectures and pointing out some interesting open problems. 



II. DISTRIBUTIONAL RECURSION 



A. 



Definitions 



We denote by V and E the vertex and edge sets of the infinite A;-ary tree by its root and 
by Vi the set of generation-^ vertices (\Vi\ = k e ). The broadcast process generates a random color 
configuration X_ = {Xi : i G V} with Xi G {1, . . . , q}. The root color Xq G {1, . . . , q}, which 
we also call the transmitted color, is uniformly random. Then, given the values of X up to the 
l-th generation, the values at the {£ + l)-th generation are conditionally independent. If a vertex 
in the £-th generation has color y, the probability that a vertex connected to it in the (£ + l)-th 
generation has color z is ir(z\y). 

We shall denote by 2Le the configurations of colors at the I th generation, and by Y? the configura- 
tion up to the £ th generation (i.e. Y_ = {^oj^Ij • • • i^j})- The probability distribution of X_g, con- 
ditioned to the choice Xq = x of the root color will be denoted by B x t \x l ) = P{2^ = Xi\Xq = 

Suppose now that the configuration of colors at the l-th. generation, x^, is given. We denote by 
rji(y) the probability that the root had sent the color y, given X t : 



rji{-) is a probability distribution over {l,...,g} (i.e. rje(y) > and YlyVe(y) = !)• We shall 
denote the space of such distributions as 97l g . In order to emphasize the dependency of rji upon the 



that, given the colors received at the £ generation, %(■) constitutes a sufficient statistics for the 
root color x. In other terms, given X_^ = xj>, there is no loss of information in computing %^(-) 
and then guessing Xq from r] x (•). 

Since X_g is chosen randomly according to the broadcast process, %(•) is a random probability 
distribution, i.e. a random point in 9Jl q . We denote by Qx\v) hs distribution 1 conditional to 
the broadcast being started from Xq = x, and call it the 'distribution at the root'. Hereafter a 
distribution Q over 9Jl q will be said trivial if it is a singleton on the uniform measure rj (defined by 
Vx : fj(x) = l/q). Clearly the reconstruction problem is solvable if and only if the large £ limit of 
Qx^ is non trivial. 

1 Notice that we adopt here the standard physicists convention: we carelessly denote probability distributions by 
their densities even if such densities do not exist. We shall also denote by J f(rf) dQi (77) the expectation with 
respect to such a distribution. The fussy reader can easily translate all the formulae below in the standard 
probability language. 



m (y) = F[X = y\X l = x l }. 



(6) 



configuration received in shell £, x 



; £ , we shall sometimes write %(y) = rjx e (y)- It is easy to realize 
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There are several ways of characterizing quantitatively the large I behavior of Q x • We shall 
consider below two parameters In = I(Xq; X?) and tyg, which are defined by: 

h = -J^J lo §2 =|| dQ(f)(r?) , ^ = ^ E / " ^)] d ^) • (7) 

If gives the number of information bits that can be transmitted reliably per network use. *f>£ 
is the probability that the reconstruction is successful when the receiver guesses color y with 
probability rji{y), minus the the same probability when the receiver guesses uniformly. These are 
non-negative quantities and can be shown to be non-increasing functions of I. We furthermore let 
loo = lim^oo I l} and = lirn^oc 

The tree reconstruction problem can be rephrased by saying that the problem is solvable if and 
only if loo > (or, equivalently, > 0). For instance, for the ferromagnetic Potts channel, the 
threshold £^~(k, q) is the supremum of the values of e such that 1^ > 0. 



B. Merging rooted trees 

How does one compute the distribution rj(y) on the root, given a boundary = Xjp. Using 
the tree-structure, this can be done iteratively by a dynamical programming procedure starting 
from the leaves. Suppose that at some point in this iteration we have determined the probability 
distributions 771 ( •),...,%(•) of the k vertices in the tree which lie above a given vertex (see Fig. [2 
left). Then the probability rj(y) that this vertex had color y during the broadcast is given by: 

^) = -?7^n(x>teMw(w)] . ^(M)^En(E^^)^)) • ( § ) 

W'i=l\ K =l / Fl.=lU. / 

This equation defines a mapping between distributions in 9Jt g : given k distributions 771 , . . . ,rjk, one 
generates a new one r\ = F(r/i, . . . , rjk). Iterating this mapping downwards from the leaves down to 
the root, one can derive the the conditional distribution of the transmitted symbol. 

Equation (JSJ naturally induces a recursion equation for the distribution Q x ■ Consider the 
reconstruction of the root in a rooted tree with t + 1 generations (see Fig. 21 right). This graph 
is formed by k subtrees rooted in the vertices 1, . . . , k, which are all joined to the root 0. Each of 
these subtrees gives an instance of the reconstruction with I generations. Therefore: 

Qi i+1) (v)= E n*(*il*) / S[ V -F( m ,...,r, k )] f[dQ®(rH), (9) 

x 1 ...x k i=l i=l 

where <$[■■■] represents a Dirac delta function on dJtg. In words, in order to generate 77 ( • ) with dis- 
tribution Qx ■, one can proceed as follows. First draw k independent colors x\,...,Xk from 
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FIG. 2: Left: a pictorial representation of the mapping rj = F(r?i, . . . , T]k) defined in (JSJ). Here k ~ 2, a 
straight line corresponds to the channel 7r, a wiggly line arriving on a vertex yj corresponds to a weight 
rjj(i)j). Right: a pictorial representation of the recursion Q. A triangle of depth r rooted on variable Xj 
denotes Qx-(ri r ) 

the distribution ir(-\x). Then, draw Tji,...,rjk independently with distribution, respectively, 
Qxl, ■ ■ ■ , Qxl- Finally, let n = F(r]i, ...,%). 
The initial condition is 

Q x 0) (r l ) = 8[r l -S x } , (10) 

where S x is the distribution in DJl q which has weight unity on color x (it is given by 5 x (y) = 1 if 
y = x, and 5 x (y) = otherwise). The equations Q and fully characterize the distributions 
Qx^ ■ The whole reconstruction problem amounts to understanding the large t properties of these 
recursions. 



C. Unconditional distribution and symmetry properties 

While Qf gives the distribution of %( ■ ) (defined in Eq. ©) conditional on the transmitted 
color being equal to x, it is equally interesting to consider the unconditional distribution. We will 
denote it by Bayes theorem implies the following relation between Q$ and 

Q ( i\n)=qr ] (x)Q^{i 1 ). (11) 

This is in fact a rephrasing of the identity 

nVX, =v\Xo = x} = f{Xq - x} . (12) 

An alternative (analytic) proof can be obtained writing Q%' in terms of Bx\x£), the probability 
that the output of the broadcast process at generation I is a^, given that the transmitted color is 
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x: 



(13) 



It is then easy to show that, if Ao + Ai + • ■ ■ + A g _i = 1, then the expectation value 

i](0) Xo ■■■7}(q- l) Xq - 



rj{x) 



(14) 



does not depend upon x. This in turns imply that Q x can be written in the form (|llj) where 

is a distribution which does not depend on x (the normalization can be found by summing over x). 

If the channel is symmetric with respect to permutations of the colors (as is the case for Potts 
channels), the distributions Qx an d inherit the same symmetry. More precisely, given a 
permutation acting on the colors a G S q , and a distribution r) £ 9Jtq, let rf be the permuted 
distribution defined by rf(x) = rjiaix)). Then, for any permutation a, QxM = Q^) \ ( r ] a )j an d 



Q ii) (v a ) = Q w (v). 



(15) 



A distribution satisfying condition (|15|) will be called 'symmetric'. 

Let us finally notice that the parameters introduced in Sec. [H] to measure the amount of infor- 
mation on the transmitted color available at the £ th generation can be expressed in terms of the 
distribution 



D(r,\\rj) dQ%) , <^ = W [v(x) -rj(x)] 2 dQ^(r,) . 



(16) 



Here we use the standard notation for the Kullback-Leibler distance 



CT9l| D(ri\\rj) = 



Ylx r }( x )^ S2[vi x )/ r l( x )]- I n deriving the second of these expressions, we used the fact that 
f rj{x) dQ^(rj) = rj(x) = - which follows from fjl 1|) . 



D. Recursion for the unconditional distribution and spin glass correspondence 

The recursion relation Q on Q$ implies the following recursion for the unconditional distri- 
bution: 

Q {l+l \v) = q"- 1 J <{m}) 5 Iv - Hm, ■ ■ ■ ,m)\ J[dQ®{vi) , (17) 

i=l 

where z({rji}) is defined as in Eq. (JHJ). The initial condition (|lUj) converts into Q (??) = 
a Yly=i $ [v( ' ) ~~ fiy( ' )]• It is also interesting to study the fixed points of this recursion, i.e. the 
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distributions Q* satisfying: 

Q*(r 1 ) = Q k ~ 1 J S[ V -F(vx,...,Vk)] II d ^)> ( 18 ) 



i=i 



Notice that any solution of this equation has necessarily expectation J r]{x) &Q*{rf) = rj(x) (this is 
proved by taking expectation on both sides). Any probability distribution over satisfying this 
condition will be hereafter said to be 'consistent'. 

The distributional equation (|18|) is well known in spin glass theory and usually referred to as 
'1RSB equation with Parisi parameter m = V (in the general 1RSB scheme the factor z({r]i}) is 
raised to a power m G [0, 1]). It is used to determine whether an associated statistical mechanics 
model is in a glass phase. We shall return to the definition of the associated model in Sec. IVII 
For the time being, we shall adopt the usual physicists criterion as a definition: We will say 
that the statistical mechanics model associated to the reconstruction problem (characterized by a 
degree/kernel pair k,n) admits a glass phase if and only if Eq. 1)18|) has a non-trivial solution. 

When considering a continuous family of kernels 7r( • | •), parametrized by a noise level e, the 
value of e where a non-trivial solution appears is called a dynamical glass transition. The result 
below implies that this coincides indeed with the reconstruction threshold (i.e. with the extremality 
threshold for the free boundary Gibbs measure on the infinite tree). 

Proposition 1 The statistical mechanics model associated with the degree/kernel pair k, it admits 
a glass phase, if and only if the corresponding reconstruction problem is solvable. 



Proof: As noticed for instance in BWOol ]. the sequence of random variables r}n{ ■ ) (not condi- 
tioned on the root color), converges almost surely to a limit • ). As a consequence, the sequence 
of distributions converges weakly to the distribution of , n 00 (- ). By taking the limit of 

Eq. lfT7|) (and noticing that F(r/i, • • • ,%) and z({r]i}) are continuous and bounded) we find that 
q(°°) must satisfy the fixed point condition ()18j) . If the reconstruction problem is solvable, then 
q(°°) i s non-trivial and therefore, according to our definition, the pair k, tt admits a glass phase. 

Conversely 2 , let Q* be a non-trivial solution of (|18|) . Following (|11|) . define the distribution 
Qx( 7 l) = Q 7 l{ x )Q*{v)- Because of the above calculations, the q distributions Q*, x G {l,...,q} 
are a fixed point of the recursion Q. We will now show that they can be used to reconstruct the 
transmitted color from the output at generation £, with probability of success independent of i and 
strictly larger than 1/q. 



2 The idea of the converse is due to James Martin who kindly agreed to let us publish it here. 
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The reconstruction procedure goes as follows. Suppose that the broadcast has generated the 
values Xi = x% for i £ Vg. For each vertex i, generate rji from the distribution Q*.. Consider now 
a vertex a G V^-i, connected to ai, . . . , a& in V£. Compute r/ a = F(ry ai , . . . ,r/ afc ), where F(- • • ) is 
defined as in Eq. iJSjl. Proceeding downwards from the leaves to the root, this allows to construct 
t]q. At this point the transmitted symbol can be guessed, for instance, by choosing Xq = y with 
probability proportional to r)o(y). 

We claim that for each vertex j S T ).(£), and conditional to the broadcast having produced 
Xj = Xj, the r)j(-) provided by the above procedure is distributed according to Q*.. This in 
particular implies that the probability of guessing correctly the root color is 

i W r,{x) dQ*(rj) = £ I r,(xf dQ*(r?) > - . (19) 

^ x J x J " 

The claim is proved by induction starting from the leaves and proceeding downwards to the root. 
It is true by construction for the vertices of the last generation. Assume it to be true up to 
generation r and consider a site a in generation r — 1 connected to ai, . . . , a& in V r , under the 
condition X a = x a . It is clear that the distribution of rj a is obtained through the recursion Q 
(with Qx} replaced by Q Xa ), and since Q% a . is a fixed point of this recursion, this proves the claim. 
□ 



III. VARIATIONAL PRINCIPLE 



A. The general principle 



Here we establish a variational principle from which the fixed point equation (|18|) for the dis- 
tribution at the root can be deduce d. We shall not try to explain here its physical origin, which is 



related to spin glass theory 



MPOll ]. but just discuss the relation with the reconstruction problem. 



Throughout this section we use the notation n(x\y) = n(y\x) = ir(x,y). Given a distribution Q 
over 9K q , we define its complexity as 

— Y~ I W e (r]i,ri2) dQfa) dQfa) + / W v ( m , . . . ,r/ k+1 ) 



S(Q) 



fc+i 
I dQfa 



(20) 



where 



W P = 



J2x u x 2 m{xi)m(x2)7r{xi,x 2 ) 

T, X1 ,X2 rj{xi)rj(x 2 )ir(xi,x 2 ) 

HxUiExi^i^^i) 
Ex UiJ2 Xl V(Xi)TT(x,Xi) 



log 



T / x 1 ,x 2 v(xi)v(x 2 )tt(x 1 ,x 2 ) 



log 



Ex 1 ,x 2 r j(xi)v(x2)Tr(xi,x 2 ) 

J2xUi TtXjViiXiMXiXj 
J2 x UiJ2 Xl fj(Xi)TT(x,Xi] 



(21) 
(22) 
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The complexity is interesting for the reconstruction problem because of the following remark: 

Proposition 2 LetQ* be a distribution overdJt q which satisfies the fixed point equation Then 
Q* is a stationary point of the complexity S(- ). More precisely, given any consistent distribution 
Q over Wl q , define E*(i) = S((l - t)Q* + tQ). Then 



dt 



0. 



(23) 



t=o 



Proof: This proposition is a direct consequence of Lemma |2] in Appendix ^] (the proof consists 
in explicitly computing the derivative of £*(£) and checking that it vanishes under the fixed point 
conditions). □ 



The complexity £ can also be written in terms of the conditional distributions Qx(v) = 
qn{x)Q(rj). Define p{xi,x 2 ) to be the marginal distribution of two neighboring variables on the 
tree: p(xi,x 2 ) = Tr(xi,x 2 )/q. Similarly, let p(x\ . . . a^+i) = [Y^ x ir(x, Xi) . . . ir(x, xj.j r i)\/q, the dis- 
tribution of k + 1 variables with one common neighbor. The complexity is then given, in terms of 
Qx(v), by: 

k + 1 



where 



S (Q) = Y~ X^( Xl > X2 ) / W e(»7l>>72) dQxi(^l)dQa. 2 (7/2) + 

XI ,X2 

n k + 1 

+ y^ j p(xi ■ ..Xk+i) / W Y (rji, . . . ,rjk+i 

{Xi} J i=l 



W P 



log 
log 



T,xux2 Vi( x i) r l2(x 2 )ir(x 1 ,x 2 ) 
E^Lxa rj(xi)fj(x 2 )7r(x 1 ,x 2 ) 



(24) 



(25) 
(26) 



B. Implications on reconstructibility 



Experience from spin glass theory, and the physical interpretation of the complexity, suggests 
the following conjecture: 

Conjecture 1 Consider the reconstruction problem for the k-ary tree and a reversible channel 
ft{y\x) = n(x\y). If there exists a consistent distribution Q tT over 9Jl q , such that S(Q tr ) < 0, then 
the reconstruction problem is solvable. 
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Let us give here a few comments in favor of the plausibility of this conjecture. Notice first that, if Q 
is trivial, then E(Q) = 0. Let P(9Jtq) denote the space of consistent probability distributions over 
9Jl q . Suppose that there exists Q tT with T,(Q tr ) < 0. Consider now the distribution Q* G P(VJl q ) 
such that the complexity is minimal. Of course S(Q*) < T,(Q tT ) < and therefore Q* is non-trivial. 
If (i) Q* is a stationary point of the complexity, and (ii) the stationary points of in P(SD f l (? ) 

coincide with the solutions of the fixed point equation (|18|) . then the existence of Q tr implies that 
the reconstruction problem is solvable. Point (i) amounts to banishing the possibility that Q* is 
on the 'border' of P(9Jt q ). Point (ii) is a stronger version of Proposition |21 

Notice that a priori one could formulate a similar conjecture with a Q tT having S(Q tr ) > 0, 
replacing 'minimum' with 'maximum' and 'negative' with 'positive' in the above. It is easy to find 
counterexamples showing that this 'reverse' conjecture is false. The reason is probably that the 
distribution Q* maximizing is on the border of P(DJl q ), and therefore (i) does not hold. 

Assuming Conjecture ^ to hold, it implies a simple variational technique for proving that re- 
construction is possible. Just consider an explicit finite-dimensional family of distributions 
depending on some parameters fi G ~R d , and minimize S(Q M ) over fi. If the minimum is negative, 
then reconstruction is possible. We will apply the variational principle in this form in the next 
Sections. In the rest of this Section (and in Appendix EJ) we shall prove the principle for a special 
family of kernels 7r( • | • ) including the antiferromagnetic Potts channel. 

We define a kernel ir(y\x) = tt(x, y), x, y G {1, ...,(?} to be 'frustrated' if it can be decomposed 
as Tr(x,y) = 7r* — ir(x,y) where tt* G R is a constant and 7r(x,y), x, y G {1, . . . , q} is a positive- 
definite matrix. The antiferromagnetic Potts kernel is a particular instance of this family, with 
7T* = e/(q — 1), and tt(x, y) = | A2 1 S x ,y where A2 = 1 — qe/ (q — 1). 

Our basic result is the following. 

Lemma 1 Let vr( • , • ) be a frustrated kernel and Q* a consistent distribution over 9Jt q which is not 
a solution of the associated fixed point equation Uty) . Then there exists a consistent distribution Q 



over 9Ji q such that: 



^£((1- t)Q*+tQ) 



<0. (27) 





The proof of this statement is postponed to Appendix Here we limit ourselves to proving that 
it implies the desired principle. 

Proposition 3 Conjecture^ holds true in the case of frustrated kernels: Let vr( • , • ) be a frustrated 
kernel, and £( • ) the associated complexity function. If there exists a consistent distribution Q over 
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dJlq such that Ti(Q ) < 0, then the reconstruction problem is solvable. 

Proof: Let S m j n = inf S(Q), the inf being taken i n P(VJl q ). Since this space is subsequentially 
compact with respect to the weak topology jshi9(j | . and X is continuous with respect to this 
topology, there exists a consistent distribution Q*, such that E(Q*) = S m ; n . Because of Lemma 
n Q* is a solution of Eq. (|18j) . Furthermore E(Q*) < E(Q tr ) < and therefore Q* is non-trivial. 
The result is a consequence of Proposition^ □ 



C. An application to Potts channels 



Here we describe a simple family of distributions which can be used variationally when studying 
the Potts channels. We will show in the next sections that, in spite of its simplicity, it leads to 
rather accurate results. 

The family is indexed by a single real parameter [i G [0, 1]. We shall denote by the corre- 
sponding distribution and will write, with some abuse of notation, S(/i) = S(Q M ). The distribution 
Qn attributes equal weight 1/q to the q points in 9Jl q denoted by 7^, x G {1, . . . ,q}, defined as 
follows 



i {x \y) 



1 - n \fy = x, 

fi/(q — 1) otherwise. 



Some calculus shows that X(^) = — ^±1 w e (^) + w v (/j), where 

1 

W e (fi) ~- 



--^log^- - — -.Blog-B. 



A 
B 



s 



q-1 

£ 



1 



H 1 



qe 



qe 



(l-/^) 2 + 



/'" 



q-1 



and 



1 



E 



k + 1 



z\n\ 



ni,...,n q 

e + fi qefi 



2m(1-/x) (g~2)/x 2 
q-1 + {q-lf 



z[n] log z[n] 



1 {q-lf 



fc+l <? 



E 

x=l 



e + (g- l-ge)(l-/i) 
e + (q- l-qe)n/(q- 1) 



(28) 



(29) 
(30) 
(31) 

(32) 
(33) 



the first sum being restricted to m, . . . , n q > and n\ + • + n q = k + 1. 

Let us briefly discuss how these formulae are used in the following. To be definite, we refer here 
to the ferromagnetic case, the antiferromagnetic one being completely analogous. Given k, q and e, 
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FIG. 3: The complexity for the ferromagnetic Potts channel with k = 2 and q = 7 within the variational 
ansatz described in Sec. lIII CI A negative complexity implies that the reconstruction problem is solvable. 
The three curves correspond (from bottom to top) to e = 0.250, 0.253, 0.256. The right plot is a zoom near 
fx = 6/7. The KS threshold for k = 2, q = 7 is e K s ~ 0.2510. For e = 0.250 < £ K s, is negative in the 

neighborhood of fi = 6/7. For e = 0.253 > £kS; as A* decreases from its maximum value 6/7, S(/x) is first 
positive, but then becomes negative with a minimum for fi ks 0.6, implying e r > 0.253. This behavior is 
typical of a first order phase transition. For e = 0.256, is always positive, and one cannot draw any 

conclusion. 

we compute £(//), and minimize it numerically for fx £ [0,1]. The largest value (more precisely, the 
supremum) of e such that the minimum value is negative, is denoted by £ wai {k,q). According to 
conjecture^ we expect e T (k, q) > £ var (k, q). Although we have proved it only for frustrated kernels 
(which do not include the ferromagnetic Potts channel), we shall loosely use the term 'variational 
bound' also in the other cases.. 

One can show that the variational bound is always at least as good as the KS one: £ v &r(k,q) > 
e Ks(k, q) by looking at the behavior of £(//) near to fj, = (1 — 1/q)- By Taylor expanding for 
(i = (1 — l/q) + 5fi, we obtain S(/x) = Cfc 5 q(e)(5^ 4 + 0(<5/i 5 ). Furthermore Cfc j(? (e) < for e < £ks(&, 0.) 
and Ck,q(s) > for e > £Ks(k,q). In Fig. Elwe plot £(//) for the ferromagnetic Potts channel with 
k = 2, q = 7, showing that the variational bound £ V ar(k,q) is strictly larger than the KS one. We 
shall discuss in the next section for which values of k, q this happens. If the variational principle 
were proved for the ferromagnetic channel, this would prove e r (k,q) > £Ks(k,q) in these cases. 

Let us notice that we do not expect the variational lower bound to be tight. More precisely, 
even minimizing it over the space of distributions over DJl q , min S(Q) becomes negative only below 
a threshold £ c {k, q) with £Ks(k, q) < £ c (k, q) < £ T (k, q) (in the case where £Ks(k, q) < £ T (k, q)). Our 
numerical simulations confirm this expectation which is motivated by the physical interpretation 
of the complexity. 
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100 



FIG. 4: The information (in bits) that can be transmitted reliably through a k-ary tree network of g-ary 
symmetric channels (ferromagnetic Potts channels), as determined with the population dynamics algorithm. 
Here k = 2, q = 15 and the noise parameter is (from top to bottom) e = 0.333082, 0.308057, 0.282444, 
0.256422. We used populations of size M — 10 5 , and averaged over 10 runs. 

IV. THRESHOLDS FOR THE FERROMAGNETIC POTTS CHANNEL 



In order to determine reconstruction thresholds numerically, we simulate the recursion @, by 

(£) 

representing the distributions Q x through a large enough sample. We will estimate reconstruction 
to be possible if the sample does not concentrate, for i large around the point r\. 

This procedure is very similar to the 'po pulation dynamics' method used to solve similar equa- 



tions in spin glass theory 



TACAjJ 



MPOlj ]. We work with q samples ('populations') P^' , . . . Pq l \ 
each containing M points rji £ i E {1, . . . , M} (i.e. M vectors rji(x), x G {l,...,q} with 
Vi( x ) — an d ^2 x Vi( x ) = !)■ The population represents an i.i.d. sample from the distribution 



Q$ . The population P x +l \ x G {1, . . . , q} is computed, for each I > as follows. 

• Choose k iid colors x\,...,X}. with distribution ir( • \x). 

• Choose k vectors 771 , . . . with rji uniformly random in P Xi . 

• Compute 7/ = F(?7i, . . . , 7/fc) according to l[5|). 

• Store this new r\ in the population P x +1 \ and repeat until the population contains M 
elements. 

This whole cycle is repeated until the populations Px become stationary (by this we mean that 
their moments no longer depend on €) within some prescribed accuracy. 

Reconstructibility can be monitored by computing the parameters 1^ and *$>i on in the popula- 

(£) 

tions P x . If Ie,^e -> as f -> 00, then we estimate that reconstruction is not possible. If they 



17 



£ r/£KS 



1.03 
1.03 
1.02 
1.02 
1.01 
1.01 
1.00 
1.00 
0.99 



1.2 












m 






m 


1.1 


m x 


























1 


□ 






X 










50 


100 












m 










X 

















10 



12 



FIG. 5: Ratio between the reconstructibility and the KS thresholds e T (k,q)/eKs(k,q), for the ferromagnetic 
Potts channel and k = 2. Squares correspond to the numerical determination of £ T (k,q) and crosses to the 
variational lower bound £ va i(k, q). The inset refer to larger number of colors (up to 100). 
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TABLE I: Thresholds (numerical results and bounds) for the ferromagnetic Potts channel. The recon- 
struction threshold e r , whose numerical estimate is shown in the first column, satisfies the rigorous bounds 
£r > £ks, £r > £aig, and £ r < £f^ p . The 'algorith mic bou nd' e a i g is computed by analyzing reconstruction 
through recursive majority along the lines of Rcf. Mos98 . The variational principle (that is not proven for 

this ferromagnetic channel would imply e T > £ vaT . The symbol means that the corresponding bound 

does not provide any information. 



instead converge to a finite value, we take this value as an estimate of loo, ^oo- Figure 0] shows an 
example of such a calculation. Reconstructibility thresholds are determined by repeating the same 
experiment for several values of the channel noise e. 

Numerical simulations clearly show that the reconstructibility and Kesten Stigum threshold 
coincide for q = 3 and q = 4. We checked this to be the case for q = 3 and k = 2-7, 10, 15, 20, 30, 50, 
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TABLE II: Thresholds (numerical results and bounds) for the antiferromagnetic Potts channel. The recon- 
struction thresho ld e r , w hose numerical es timate i s shown in the first column, satisfies the rigorous bounds 
£r < £k s (from KS66a ), e T < e a i g (cf. |mos98| i. e r < e var (from Proposition EJ) , and e T > e^p (from 
MP03bJ). The symbol means that the corresponding bound does not provide any information. 



and q = 4 and k = 2, 3, 5, 10, 15, 30 and expect it to be the case generically, at least for k not too 
large. When this is the case, the order parameters 1^, decrease continuously and vanish at 
e v (k,q) = e KS (k,q). 

For q > 5 we always find e r (k,q) > £ks(&><?)- I n these cases /00(e) | J* > 0, ^oo(e) | ^* > 
as e | £r(k,q). In spin glass language, the transition is discontinuous: we refer to next Section 
for some illustrations. We report our numerical results in Table [I] This table also con tains th e 
variational lower bound e va _ r (k,q) < e r (k,q), as well as the upper bound derived in 
£ T (k,q) < e^ p (k,q), where 



MP03bl] : 



'MP 



(k,q) = (q-l) 



(2 - q + 2kq) =p y/(2 - q + 2kq) 2 - Ak(k - l)q 2 



2kq 2 



(34) 



In Fig. |H]we plot the thresholds as a function of q for k = 2. 



V. THRESHOLDS FOR THE ANTIFERROMAGNETIC POTTS CHANNEL 

In Table [H] we present our numerical results for the reconstruction thresholds of the antiferro- 
magnetic Potts channel in the cases in which it differs from £ks> together with the bounds. 
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e e 
FIG. 6: Asymptotic complexity Eqo, information capacity 1^ and conditional variance as a function of 
the noise parameter for the antifcrromagnctic Potts channel. On the left: typical continuous reconstructibil- 
ity transition, q = 3, k = 6. On the right: typical discontinuous transition, q = 4, k = 9. 

One distinctive feature of this channel is that, even in the limit e — > 1 reconstruction may be 
impossible. For any given q > 3 reconstruction becomes possible only for k > k*(q). Numerically 
we found fc*(3) = 5, fc*(4) = 8, fc*(5) = 13, fc*(6) = 17. In fact the case e = 1 has a special interest. 
In this case the configuration produced by the broadcast process is distributed according to the 
free boundary Gibbs measure for proper colorings of the (infinite) tree TV Our numerical results 
imply that this measure is extremal only for k < k*(q), with k )f (q) as above. Using the variational 
principle (which in this case is proved, cf. Proposition 0), we can show that /c*(3) < 5, fc*(4) < 9, 
fc,(5) < 13, fc*(6) < 17... 

For q = 3 we found the reconstructibility threshold to coincide always with the KS threshold. 
This was checked for k = 4-7, 10, 20. The parameters 1^ and are continuous functions of 
e vanishing at £ks- An example is provided in Fig. left frame. For q > 4 and k > k*(q) the 
transition at the reconstructibility threshold is discontinuous, cf. Fig- El right frame. Table ITT1 gives 
the values of /*, ^ and = lim e ^ £r S(Q'- 00 ' 1 ) (in the ferromagnetic channel, this number is so 
small that it cannot be measured reliably in the numerics). Most of the remarks made for the 
ferromagnetic channel apply to this case. 
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VI. RELATION TO SPIN GLASS THEORY 

In this section we explore the link between reconstruction and spin glass theory. For simplicity 
we keep to the Potts channel, but the discussion can be easily generalized. Consider a configuration 
Y_ L generated by the broadcast along a finite rooted tree T^(L) with L generations, starting from 
a uniformly random symbol in {1, . . . , q} at the root. As we saw in the introduction, Y_ is an 
equilibrium configuration of the Potts model with free boundary conditions on T/%(L), i.e. is 
distributed according to the Boltzmann law for the energy function ©. The coupling J of this 
model is given by e~@ J = ^ g _ 1 ^ 1 _ e ^ , and is ferromagnetic (resp. antiferromagnetic) if J > (resp 
J < 0). 

Once the broadcast process has fixed the variables on the boundary at distance L from the root, 
the reconstruction problem can be phrased in terms of the conditional distribution P{Ao = x\X_ L = 
x L }. The distribution of the first L— 1 generations given the received symbols, ¥{Y = x_l}, 

is also given by Boltzmann law for the energy function However the boundary condition is 
now given by the received symbols. One fundamental reason why reconstruction is related to a 
spin glass problem is that this boundary condition tends to frustrate the system, in the sense of 
creating conflicting constraints. 

It is well known that on trees, frustration comes only through the choice of boundary conditions. 
Here we discuss the spin glass phase induced in the Potts model on the tree by various possible 
choices. We shall first show how a 'naive' choice of boundary conditions leads to a simple replica 
symmetric recursion relation. Then we show how well-chosen self-consistent boundary conditions 
lead to the correct 1RSB fixed point equation of reconstruction. Finally we discuss the explicit 
realization of the corresponding spin glass model as a model of Potts spins on a random graph (not 
a tree). 

A. Boundary conditions with independent spins 

As before, we call X_ L the set of all spins in the L th generation. A boundary condition (BC) is 
a probability distribution on these spins. One first possibility is when spins on the boundary are 
independent random variables: a given spin Xi, i E Vl takes value X{ with probability i]i{xi). The 
overall distribution is therefore IlieVL O nce a se * °f Vi( ' )' s i s given, the partition function 
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of the tree is obtained as: 



ZU{Vi}ieV L ) = Y1 II Tr(xi,Xj) ~[\i]i(xi) , (35) 

y L (i,j)er k (L) i€V L 



and the Boltzmann distribution is 



(y L ) = ^r4rT II <*i>*j)T[vi(*i)- (36) 



{r?i}v» i Z L ({ Vi } 



We have still the free dom o f chosing the rj j{ . ). One simple possibility would be to take them 



identical: r)i(-) = r)(-) 




Per84l . lPer83| . but in order to have a disordered and frustrated 
problem one can choose to sample the independently from a symmetri c distribut ion p(°\r]). 
This definition of a spin glass problem on a tree was adopted for instance in |cCST8^ | in the Ising 
case (q = 2), where each of the boundary spins was fixed to ±1 independently. In our formulation, 
this corresponds to the choice p(°>(r]) = 5 (6 [77 ( ■ ), <5+i] + S [r]( • ), <5-i]). 

The recursive procedure for merging rooted trees applies in the same way as in Sec. Ill Bl 
Consider the marginal distribution on the first L — £ < L generations, P(y ). It is clear that 
this has the same form as in Eq. (|36|) . with some new 77- ( • ), i G Vl-i- When one generate BCs 
randomly as described above, the i] ■ are iid random variables with common distribution P^\ij). 
A little thought shows that P^ (77) is related to the one in the shell just above by: 

PV +1 \V)= [ S[ V -F( Vl ,..., Vk )] HdP^i^). (37) 

i=l 

Notice that in each shell, is symmetric. 

The marginal distribution at the root of the tree T^(L), under the Boltzmann law (|36jl . is a 
random variable with distribution p( L \rj). In this model, the existence of a spin glass phase is 
characterized by a non trivial limit of P^ L \ij) — > p(°°\r]) as L — > 00. Such a limit solves a fixed 
point equation corresponding to 1|37|) . 

The reader will notice that Eq. (|37|) is similar to the reconstruction equation ()17|) . with one 
crucial difference: the 'reweighting' factor z({r]i}) in H17|) is absent here. In the spin-glass jargon, 
Eq. (|37|) is the 'replica symmetric' (RS) equation, while Eq. (|17j) is the 1RSB equation with Parisi 
parameter m = 1. 

We want to argue that the model defined by Eq. I)36|) . with iid r/j's in not a 'good' model of spin 
glass on a tree. Technically this is seen from the fact that its glass phase is a RS one, while the 
spin glass models on graphs with loops typically show RSB. Fundamentally, the drawback of this 
model is precisely that it neglects correlations between spins on the boundary. As we will discuss 
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in Sec. IVI CI such correlations are necessary in order to study the existence of many pure states, a 
distinguished mark of spin glasses on graphs with loops. 

A minimalistic way of introducing such correlations is to keep uniquely those correlations in- 
duced by the tree itself; this is precisely what is done in reconstruction, as we now discuss. 

B. Self-consistent boundary conditions 

It is clear that the broadcast/reconstruction process generates correlated BCs. By this we 
mean that the conditional distribution F(Y_ L ~ 1 = y L ~ 1 \X L ) has still the form (|36|) but the r]i are 
no longer independent. More explicitely, the ^'s, i G Vl have distribution: 

H{mUv L ) = ±- Z L {{ m } ieVL ) J] Q(°>fo) , (38) 

where Z({r)i}) is the partition function ()35|) of the tree Tfc(L) with BC {?/«}, and Q (v) is the 
uniform distribution on the q 'corners' of the simplex rj(x) = 6 x>r , r G {1, . . . , q}. 

We can analyze the system with BC (|38|) as we did in the previous section for uncorrelated BCs. 
Consider, as before, the marginal distribution of the first L — £ < L generations of the tree. It also 
has the form (|36|) and the new r^'s at distance £ retain the same correlation structure. Their joint 
distribution is 

HHhevt) = i- Himhm) IT ^ ( %) ■ ( 39 ) 

Finally, the distribution is related to through the recursion (|17|) that we found when 

discussing reconstruction, with the correct reweighting factor. Since we took Q(°)(t?) = Q (0) (??), 
this implies Q^\rj) = Q^(jj) for any I > 0. 

It is interesting to define the spin glass problem on the tree associated with the non-trivial 
fixed point of Eq. (j!8[). This just amounts to generating the BCs from (|38[) using = Q* . 
This problem has the virtue of being statistically translation invariant 3 (although, for any given 
realization, the resulting Gibbs measure is not translation invariant). In particular the properties 
of a spin don't depend on its distance to the root. 



3 Provided one replaces the rooted tree (with a root of degree k) with a regular Cayley tree (with all the vertices of 
degree k + 1). 



23 




FIG. 7: The basic cavity recursion. 



C. Spin glass on the Bethe lattice 



While the previous definition of a spin glass on a tree is perfectly correct, it is clear that a lot 
of the physics has been put into the choice of the BC distribution. This is necessary because of 
th e crucia l role of BCs on trees. An alternative definition of the Bethe lattice spin glass, proposed 



m 



MPOll ]. is to use, instead of trees, random graphs which have a tree structure on finite length 



scales. Let us consider for instance the problem of N Potts spins on the vertices of a random 
regular graph Qn with degree k + 1, with pairwise interactions given by the kernel ir(x,y) The 
partition function of such a model is 



Z= ^2 n(xi,Xj) 

X!,...,x N (i,j)eQ N 



(40) 



In any finite neighborhood of a randomly chosen node i, the local structure 4 of Gn is (with high 
probability) the one of a regular tree with degree k + 1. In fact, the shortest loop through i is 
typically of size log N which diverges when N — > oo. This setting is interesting for two reasons: (i) 
Loops, although large, can create some frustration; (u) The system is approximately homogeneous 
(unlike on a regular tree, where vertices on the boundary have a neighborhood very different from 
the others). 

Spin glasses on random lattices with a local tree-lik e structu re have been the object of many 
studies in recent times. The cavity method of MPOll . |MP03a] is an iterative procedure which 



exploits the tree-like structure. Here we shall just mention some of its main results without jus- 
tification, the aim being to clarify the correspondence between the spin glass model on a random 
graph and the reconstruction problem. 



4 By this we mean the subgraph wit hin any fixed distance from i. The property described here can also be phrased 
in terms of local weak convergence |AS04| 
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In the cavity method one first considers the graph, rooted in a site i, obtained by cutting the 
edge to one of its neighbors, cf. Fig. The marginal distribution of the root, rji^,j(xi), 

with respect to the model on the 'amputated' graph, is then written in terms of the distributions 
r li~*i{ x i) where I are the neighbors of i different from j. It is possible to write such a recursion 
only if the variables xi, in the absence of the edges become uncorrelated in the large N limit. 

Such a property is made possible by the local tree-like structure, but also requires a fast decay 
of correlations in the graph. This is expected to happen either when the system admits a single 
Gibbs state, or when the Boltzmann measure is restricted to a pure (extremal) Gibbs state 5 . In 
the first case, the problem is described by a unique distribution rj(x), and rji-^j = r] for all directed 
edges i — > j. This distribution is a fixed point of satisfying thus r\ = F(rj, ... ,77). It is called 
the 'paramagnetic' or 'liquid' phase. In the case where there exist several pure states, the recursion 
holds when the measure is restricted to one pure state a: on a given (large) graph one could thus 
generate a set of 'messages' 77^ -(xj) for each state a. Notice that, for a given a, the messages 
now depend explicitely on the edge: the measure is no longer uniform, but it is modulated. The 
1RSB cavity method assumes that there exist exponentially many such pure states, the number 
N(f) of states with free energy density F a /N = / is written in terms of the complexity function 
£(/) as N{f) = exp(iV£(/)). In such a case one can perform a statistics in the space of pure 
states, by introducing, for each edge the probability Ri-+j{rj) that the message r]"^ = Tj, 

when a is chosen randomly with a weight proportional to the total Boltzmann weight of state a. 
After performing this average over states the various edges become again equivalent, and one finds 
that the distribution Ri_j(r]) = Q* satisfies exactly the 1RSB fixed point equation QlKj). So there 
exists a 1RSB glass phase if and only if this equation has a non trivial symmetric solution. Notice 
that this equation can also have other non-symmetric solutions. For instance in the case of the 
ferromagnetic Potts channel, at low enough temperature there is a solution where Q* is peaked 
on a 77 with a ferromagnetic bias, but it does not satisfy the symmetry property that we impose 
for the study of the glass state. In such a system the glass solution exists, but it is not realized 
on a random graph: the system will transit to a ferromagnetic phase. On the contrary in some 
other cases the glass phase will be realized. For ins tance we expect this to be the case for the 



MPWZ02 



BMP+03 ]. 



antiferromagnetic Potts model on the random graph 

The tree reconstruction problem on the one hand, and the spin glass on a random graph on 
the other, thus naturally lead to the same equations. Some aspects of this correspondence call 



5 The definition of extremal Gibbs state on a finite graph goes beyond the scope of this paper. 
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for a better understanding. Consider the model on the random graph defined in Eq. (|4U|). Let 
us suppose that it has several pure states, and that the 1RSB cavity solution of the problem 
is correct. Now isolate around an arbitrary point the set of all its neighbors up to distance I. 
Generically it is a tree Y k {€). The vertices outside this tree create some boundary condition on 
the leaves of this tree, depending on the pure state a that we are considering. We have found 
that the statistics of these BC on the pure states corresponds to the statistics of the boundaries 
in the broadcast/reconstruction, and both are described by the distribution Q*. In spin glass 
theory (within 1RSB) one can count the pure states through the computation of the complexity 
function £(/). It would be very interesting to have an interpretation of this function in terms of 
the reconstruction problem. 



VII. GENERALIZATIONS 



So far we have focused on fc-ary trees whose links corresponds to identical copies of the same q- 
ary channel satisfying the symmetry condition ir(y\x) = 7r(x\y). However, none of these hypotheses 
is crucial to our approach. In this section we define a considerably more general context, and sketch 
how to adapt the above formalism to this case. Some cases of broa dcast through non regular 
trees, o r with asymmetric channels have been considered for instance in 



EKPSOO 



MosOl 



MP03b 



MarO.'fl ] . The present formalism encompasses all these cases and generalizes them to broadcast 
through hypergraphs. 

We consider a finite set of kernels {tt^( ■ \ ■ ) ; a = l,...,n}, each kernel describing a 
one-to-many communication channel. For each x £ {l,...,q}, and each k a -uple y\,...,yk a , 
w^iyi, ■ ■ ■ ,yk a \ x ) gives is the probability that users l,...,k a receive outputs y\, . . . ,y ka if the 
channel input was x. These kernels must satisfy the conditions 

n^( yi ,...,y ka \x)>0, ]T 7r^( yi ,...,y ka \x) = l, (41) 

»)■■•>»„ 

and x is called the parent of y\ , . . . , y ka ■ Such 'one-to-/c' communication channels can be represented 
graphically using factor nodes of degree k + 1, cf. Fig. |HJ 

Next, we define a rafidoTn tree network ensemble depending on two probability distributions q a , 
a £ {I, ... ,n} (q a > 0, Yl Qa = 1) and pi, I > (pi > 0, YlPl = !)• One (infinite) random network 
T from this ensemble is generated as follows starting from the root 0. 

• Draw an integer Iq with distribution p\. This is the degree of the vertex 
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FIG. 8: Left: a function node representing the ! one-to-fc' channel n^- a \yi, . . . , yk\x). Right: An example of 
a small random tree network, with lo = 3 

• For each a € {1, . . . , lo}, draw a a independently with distribution q a , and attach a channel 
of type a to the vertex 0. The root will transmit through such channels. 

• For each a € {1, . . . , Iq}, and each i a G {1, . . . , k aa }, associate a vertex to the i a -th output 
of channel a. Repeat the above construction for each of these vertices. 

In Fig. |H]we show a small example of such a network. We denote by T(i) the random (sub)network 
rooted at i, by T(i,£) its first £ generations (starting from i), and by X i $ the received colors, £ 
generations above i ( X_ Q £ = X e ). 

The network is used to communicate. A color xq £ {l,...,q} is chosen at the root with 
probability ^(^o) an d broadcast through the £o channels connected to the root itself. Each of 
the first generation vertices receives a corrupted version X{ € {1, . . . , q} of this color, with joint 
distribution 

lo 

H^ a \x a , l ,...,x aAa \x ), (42) 

a=l 

where (a, r) denotes the r-th output vertex of the a-th channel. In other words distinct channels 
act independently. The same transmission process is repeated at the first generation and so forth, 
through the entire network. The problem is to reconstruct the transmitted color from the output 
at generation £, denoted as x_£- 

Analogously to the case investigated in the previous sections, we say that the reconstruction 
problem is solvable if the conditional mutual information I (Xq; X_g\T) does not vanish as £ — > oo. 
Equivalently, the problem is solvable if there is a reconstruction procedure which succeeds with 
probability strictly larger than max xo (po(xo) in the £ — > oo limit. In these definitions we assume 
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the network structure to be known at the receiver (this is why we consider a mutual information 
which is conditional to this structure). 

Unlike for q-aiy reversible channels, the distribution of the color Xi received at vertex i, is not 
uniform and depends upon the vertex. We shall denote it by tfi(xi) = F[Xi = Xi\T]. In fact <fi(-) 
can be determined recursively: if j is the parent of i, and ii, ik-i the other vertices that share this 
parent, then 

<Pi( x i)=^2 ^2 ^ a) {x i ,x il ,...,x ik _ l \x j )ip j {x j ). (43) 

Let us consider the reconstruction problem. The conditional distribution of the transmitted 
color given the observation at generation £, ¥[Xq = x\2Le = X(\ can be computed recursively 
proceeding from the leaves downwards to the root as in Sec. Ill Bl In order to simplify the analysis, 
it is convenient to 'factor out' the a priori information ipiixi), and define 

r )i/ (x)=¥ i [X i =x\X i/ }. (44) 

where Pj denotes probability with respect to a modified process in which the boundary X, ^ is 
obtained from a broadcast starting from Xi chosen uniformly at random in {1, . . . , q}. Of course 
we have 

¥[Xi = x\X^\ = - ipi(x) ^(x) , (45) 

where m = ipi(x)r/i^(x) ensures the correct normalization. Notice that rji^x) depends uniquely 
on the portion of the tree above i, more precisely on T(i,£) and X i ( . 

With a slight abuse of notation, let us denote by ir^ = ir a (- a \ a £ {1, . . . , Zj} be the channels 
whose input is Xi, and by ji, . . . , ju a \ the corresponding output vertices. It is easy to derive the 
following recursion which generalizes Eq. Q 

1 h 

m,£+i(x) = {a) Yl Yl nWfa, . ..x k(a) \x) VnAxi) ■ ■ ■ Vj Ka) ,t( x k(a)) , (46) 

? W ,tT t} a=lx^...x k{a) 

where t is the distance from the leaves. The constant z({rjj t £}, {vr( a )}) is defined by the con- 
straint ^2 x r]i,i+i( x ) = 1- We shall denote the above mapping synthetically by writing r/j^+i = 
F(fe}, 

One can define two types of probability distributions associated to 77^. We first assume that 
the tree T is given, and consider the distribution of rji^ conditional to Xi = x and T. This will be 
denoted by Qx \n)- Arguing as in the case of regular trees, one derives the recursion 

QP 1) W = En^ ) fe-^(J s ) / %- F ({%M^ (0) })] W&Q'ifXm)- (47) 

{ay}a=l ^ 3 
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Next, we consider the distribution of r\m unconditional of T, which we denote by Qx\v)- This 
can also be regarded as the expectation of the previous distribution: (n) = EjQ { J' £ \r)) . Notice 
that Qx'^(t?) depends on T only through the first t generations of the subtree rooted at i T(i,£). 
Since the structures of distinct subtrees are independent, if we average Eq. (|47jl with respect to T, 
the averages factorize, yielding a recursion equation for Q^\rf): 

Qi e+1 \ V )=K^2l[^(x jl ,...x jk(a) \x) [ S[ V -F({ Vj },{^})} Y[dQ$( Vs ). (48) 

{xj}a=l j 

Here E denotes expectation with respect to the degree li and the channel types. The last expression 
is particularly convenient for numerical simulations and it is not more complex than the iteration 
© studied in the previous Sections. 

We can also consider the distributions unconditional to the transmitted color: Q^'^irj) and 
Q^(rj). The same relation as for regular trees hold in this case Qx\v) = qr)(x)Q^'^ (rj) and 
Qx\v) = QV( X )Q® (v)- ^ is easy to derive the corresponding recursions. We just write down the 
equation for the last (non-random) distribution 

z({ Vj }{^}) 

"VI- '-{VljhY' i ' 

3 

In order to discuss the correspondence with the dynamical glass transition in this more general 
setting, it is necessary to distinguish two cases. In the simplest one, the RS cavity equations for 
the associated statistical mechanics model admit the solution rji^j(x) = rj(x) for any directed link 
i — > j in the graph. Under this hypothesis it is not hard to show that Eq. (|49jl is equivalent (in 
the same sense discussed in Sec. [n| to the m = 1 1RSB equation 6 

In the general case (i.e. if 7]i^>j(x) = fj(x) does not solve the RS equations), the dynamical glass 
transition still corresponds to the extremality of the free boundary measure on an infinite tree. 
The last problem, however, cannot be formulated in the same framework as described here. One 
can still write an equation of the form (|47|). conditioned to the graph structure, as is usually done 
in statistical physics. But the average over the graph s tructure can only be performed conditioning 



(£+1) fo)=E / %-F(M,Or (a) })] II d ^)- ( 49 ) 



upon the value of rji— >, ( • ) in the RS solution 



MM05j], and so the relation between the spin glass 



problem and the reconstruction problem unconditioned to the structure of the tree, is not as simple 
as before. We shall not enter these details here. 



6 The expert will perhaps be surprised by this remark since it is usually said that the order parameter for such 
systems is a 'measure over the space of distributions'. However it turns out that, for m = 1, the expectation of 
this measure satisfies an equation which is Eq. I)49|l . 
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VIII. CONCLUSION AND OPEN PROBLEMS 

The coincidence between the reconstruction threshold and the dynamical glass transition ex- 
plored in this paper is interesting from several points of view. 

First of all, it provides some perspective for putting on a firmer basis the theory of glassy 
systems on locally tree-like graphs developed in the last few years. As a concrete example, notice 
that the population dynamics algorithm defined in Section llVl presents two important advantages 
with respect to the procedure adopted by statistical physicists. First, in spin glass theory the usual 
prescription is to look for a solution of the fixed point equation (|18|). This poses the problem of 
the initial condition: even if a given initial condition yields a trivial fixed point, this may not be 
the case for all the initial conditions. In the present formulation the iteration is initialized with a 
very specific initial condition, and one is guaranteed that, if it converges to a trivial fixed point, no 
non-trivial fixed point exists. Second, simulating Eq. Q18j) requires a 'reweighting' of the sample 
which is usually the trickier part of the calculation. No reweighting is needed in the new approach: 
Eq. @ can be handled easily. 

Also, it provides some indication on the correctness of simple analytic and algorithmic ap- 
proaches to these systems, such as the replica symmetric cavity method, or the belief propagation 
algorithm. While it has long been known that their correctness should be related to a fast correla- 
tion decay in a model on a tree, a precise criterion has never been formulated 7 . Ous work suggests 
that the extremality of the associated Gibbs measure on a tree provides such a criterion. 

More broadly, it illustrate some subtleties of the physics of the glass transition. It is well 
known that, in a 1RSB glass transition, point to point correlations (static scattering factors) do 
not present any diverging correlation length. This paper shows that such a length can be derived 
quite generally from point-t o-set c orrelations. Indeed th e definit ion considered here is essentially 
equivalent to the one of Ref. 



MS05a 



that this length scale divergence 



BBOJ]. It was shown in Ref. 
implies a lower bound on the time scale divergence. 

Finally, statistical physics ideas can inspire new results on the original reconstruction problem. 
The most interesting such idea is, in our view, the complexity functional introduced in Section InTl 
Apart from being conceptually innovative with respect to classical techniques, it seems to provide 
by far the best rigorous quantitative estimates of the reconstruction threshold, cf. Proposition |21 



A frequently used sufficient criterion is the uniqueness of the Gibbs state on the infinite tree (see IBG05I and 
references therein). As it emerges from our discussio n, this criterion is often much stronger than needed. It is 
also interesting to recall that Tatikonda and Jordan |TJ02|1 first connected the convergence properties of belief 
propagation to the extremality of the free boundary Gibbs on a properly defined tree. 
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This is well illustrated by the results in Table ITT1 It will certainly be interesting to find a concrete 
interpretation of this object in terms of the original reconstruction problem. Also, the values of 
the channel parameter at which the asymptotic complexity E(Q(°°)) vanish have a particularly 
important role in statistical mechanics, but did not find any role here. 

There are several results that we have not been able to prove rigorously. We already formu- 
lated one such results as Conjecture^ and proved it for a particular class of channel models as 
Proposition |31 Another interesting fact which has emerged from our numerical simulations is the 
coincidence of the KS and reconstruction thresholds when the number of colors is small. 

Conjecture 2 Consider the reconstruction problem for the k-ary tree and the ferromagnetic Potts 
channel (q-ary symmetric channel) with q < 4, or the antiferromagnetic Potts channel with q < 3 
Then, there exists a A; max > 30 such that, if k < k max the reconstruction threshold coincides with 
the Kesten-Stigum threshold. 

A stronger version of this conjecture would be to require the thesis to be valid for all values of k 
(i.e. to state that k max = oo). Although we didn't find any k contradicting this stronger version, 
this might of course be due to the limitation on the values of k that we can treat numerically. 
Finally, let us single out the case of completely antiferromagnetic (e = 1) Potts channels: 

Conjecture 3 Let k*(q) be the maximum value of k such that the free boundary Gibbs measure 
for uniformly random proper colorings on the infinite k-ary tree is extremal. Then k*(3) = 5, 
fc*(4) = 8, fc*(5) = 13, fc*(6) = 17. 
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APPENDIX A: VARIATIONAL PRINCIPLE FOR FRUSTRATED KERNELS 

This appendix is devoted to the proof of Lemma^ It is convenient to introduce some notations. 
If A(x,y), x, y £ {1, . . . , q} is a symmetric matrix and rji(x), 172 (a?) , x & {1, . . . ,q} are two vectors, 
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we shall write Ar)i(x) = ^2 y A(x, y)r)i(y) and rjiAr]2 = Ey A{x, y)r)i(x)r}2(y) ■ Furthermore, if Q 
is a distribution over 9Jt q we will denote by TQ the distribution obtained by using Eq. I)18J) : TQ is 
the left hand side of Eq. (fT%)) when in the right hand side Q* has been substituted by Q. Finally, 
given rji,rj2 G we let 

%, S )=f??)fc«f??). (AD 

V wn J V wn J 

We also write r\ = Q when n has distribution Q. We first derive two simple lemmas. 

Lemma 2 Let Q* and Q be two consistent distributions over 9Jt q and = S((l — t)Q* + tQ). 
Then 

1 d£* 

o 



(fe + 1) dt 



E{A(z/,r/ 2 ) - A(z/,r ?2 ) - A( m ,r]' 2 ) + A( m , m )} 



(A2) 



where the expectation is taken with respect to the independent random variables n\ , i]2 = Q* , r]' 2 = 
TQ* and v = Q. 



Proof: Elementary calculus yields 



where 



+E 



1 dS* 



(k + 1) dt 



</>(!) -V(0), 



Ex rii=o ^(z) 



Ex rii=o ?rr ?( a; ) 



(A3) 



(A4) 



Here r?, 771, ...,% = Q* and v l = [\ — t)Q* + iQ are independent random variables. The first term, 
when integrated on t, gives the contribution E{A(t7i,t/2) — A(V, 772)} to (|A2|) . 

As for the second term, observing that nf=i nr 1i( x ) = z ({Vi}) F(?7i, • • • ,%), it can be rewritten 

as 



i/*7rF(r/i . . .rjk) 



E\z({ Vi }) 



log 

r/Trrj 
l/nFjrji ...f] k ) 



z/*7rF(r?i ...n k ) 



log 



777TT7 



+ 



({*}) 



(A5) 



Since Ei^(x) = ry(x), the second term is i-independent and does not contribute to l)A3|) . The first 



term is equal to EA(i/', 772) where r}' 2 = TQ 



□ 
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Lemma 3 Letrji,r)2 £ 9Jt<j and define 6rji(x) = i]i(x) — r/(x) . Ifir(x,y) = 7r*— tt(x, y) is a frustrated 
kernel, then 

\Sr]i7fdr]2\ < rjirrj . (A6) 

Proof: Since tt is positive definite, cjmij) = tt(x, y)4>(x)ijj(y) is a well denned scalar product. 
Cauchy-Schwarz inequality implies 

\57]itt57] 2 \ < \f (5r]i7r5r]i)(5r]2Tt5r]2) < max {(dn^drji), (^^rfe)} ■ (A7) 

Therefore it is sufficient to prove Eq. (|A6|) for drji = 5t]2 = 5rj. Let rj(x) = f](x) + 5rj(x). Since 
ir(x,y),rj(x) > 0, and ^2 x Sn(x) = 0, we have 

< r]7rr] = rjirrj + SrjirSr] = rjirrj — SrjirSri . (A8) 

□ 

We can now turn to the proof of LemmaE] In the following, given rj E ^0l q , we define 5n(x) = 
rj(x) —rj(x). Obviously we have 

A( Vl , m ) =(l- log( 1 - ^) . (A9) 

Because of Lemma 01 we can expand this expression in an absolutely convergent series 



A{i] 1 ,rj 2 ) = + 2^ Cn I , (A10) 



where C n = l/n(n — 1) > 0. If 771 = Q\ and 772 = Q2 are independent random variables with 
consistent distributions, we get 

00 

EA( m , V2 ) = Y J C n q n 4> { i ) ^ n ^2 n) , (AH) 



where 7f® n (xi . . . x n ; y\ . . . y n ) = t?(xi, yi) ■ ■ ■ 7f(x n , y n ) is the n-fold tensor product of tt, 
4>a\xi ■ ■ ■ x n ) = K{5r] a (xi) ■ ■ ■ 5rj a (x n )} are the moments of the distribution Q a: and 



E ^(xi...x nm ...y n )$\x 1 ...x n )$ ) (x 1 ...x n ). (A12) 

Cl...Xn X\...X n 

Now consider Remark |2] and take Q = TQ*. We get 

1 c\y* 

- ^— pry = E { A ( ? ?i'%) - Hv'x,m) - Hm,r&) + A( m , m )} , (A13) 
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where 771 , 772 = Q* and t?i , 77 2 = TQ*. Applying Eq. (|A11|) we get 

1 d£* 



(k + 1) dt 



= °n q n (4>T ] ~ (k {n) )^ n ^j ] ~ <A (n) ) , (A14) 
n=2 

where c^ n ) and <^j^ denote the moments (respectively) of Q* and TQ*. Since fr is positive definite, 
7?® n is positive definite as well and therefore the right hand side is a sum of non-negative terms. In 
order for this right hand side to vanish, each of the terms must vanish, which implies <ft^ = 
for each n. But, since Q* and TQ* have bounded support, this implies Q* = TQ*, which is false 
by hypothesis. Therefore the right hand side of Eq. (|A14|) is strictly positive and ^-| < as 
desired. □ 
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